Electronic device and operation method for performing speech recognition

ABSTRACT

An electronic device for performing speech recognition and a method therefor are provided. The method includes detecting a first text, which is preset for performing speaker recognition, by performing speech recognition on a first speech signal, performing speaker recognition on a second speech signal acquired after the first speech signal, based on the first text being detected, and executing a voice command obtained from the second speech signal, based on a result of performing the speaker recognition on the second speech signal indicating that a speaker of the second speech signal corresponds to a first speaker who registered the first text.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 16/545,511, filed on Aug. 20, 2019, which is based on and claimspriority under 35 U.S.C. § 119(a) of a Korean patent application number10-2018-0096823, filed on Aug. 20, 2018, in the Korean IntellectualProperty Office, and of a Korean patent application number10-2019-0090499, filed on Jul. 25, 2019, in the Korean IntellectualProperty Office, the disclosure of each of which is incorporated byreference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to electronic devices and operation methods forperforming speech recognition.

2. Description of Related Art

Recently, in the field of speech recognition, there has been anincreasing demand for research into a speech recognition device foridentifying a user's intent from his or her voice command and providinga service desired by the user.

In particular, by using artificial intelligence (AI) techniques, an AIspeaker may perform the same roles as an AI secretary by recognizing auser's voice, extracting a command contained in the voice to perform anoperation in response to the command, and outputting a result as speech.Furthermore, the AI speaker may identify a speaker and perform afunction suitable for the identified speaker by performing speakerrecognition as well as speech recognition on a speaker's conversation.

However, when both speech recognition and speaker recognition areperformed on all utterances, the amount of computation is excessivelyincreased. Another problem is that the identification of a user by theAI speaker is based on only a user's voice and thus suffers from lowaccuracy compared to user identification or authentication based onbiometric information, such as fingerprint or iris recognition. Thus, ina method of performing speaker recognition together with speechrecognition, there is a need for a technique for increasing accuracy ofthe speaker recognition and reducing the amount of computation.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providean electronic device and operation method for performing speechrecognition. Also provided is a computer program product including acomputer-readable recording medium having recorded thereon a program forexecuting the operation method on a computer. The technical problems ofthe disclosure are not limited to the aforementioned technical features,and other unstated technical problems may be inferred from embodimentsbelow.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments of the disclosure.

In accordance with an aspect of the disclosure, a method of performingspeech recognition by an electronic device is provided. The methodincludes detecting a first text, which is preset for performing speakerrecognition, by performing speech recognition on a first speech signal,performing speaker recognition on a second speech signal acquired afterthe first speech signal, based on the first text being detected, andexecuting a voice command obtained from the second speech signal, basedon a result of performing the speaker recognition on the second speechsignal indicating that a speaker of the second speech signal correspondsto a first speaker who registered the first text.

In accordance with another aspect of the disclosure, an electronicdevice for performing speech recognition is provided. The electronicdevice includes a microphone configured to receive first and secondspeech signals, and at least one processor configured to detect a firsttext, which is preset for performing speaker recognition, by performingspeech recognition on the first speech signal, perform speakerrecognition on the second speech signal acquired after the first speechsignal, based on the first text being detected, and execute a voicecommand obtained from the second speech signal, based on a result ofperforming the speaker recognition on the second speech signalindicating that a speaker of the second speech signal corresponds to afirst speaker who registered the first text.

In accordance with another aspect of the disclosure, a computer programis provided. The computer program product including a recording mediumhaving stored therein a program for performing the method according tothe embodiment of the disclosure.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a network configuration diagram showing a speech recognitionsystem according to an embodiment of the disclosure;

FIG. 2 is block diagram for explaining internal configurations of anelectronic device according to embodiments of the disclosure;

FIG. 3 is block diagram for explaining internal configurations of anelectronic device according to embodiments of the disclosure;

FIG. 4 illustrates examples of performing speech recognition accordingto embodiments of the disclosure;

FIG. 5 illustrates examples of performing speech recognition accordingto embodiments of the disclosure;

FIG. 6 is flowchart of methods of performing speech recognitionaccording to embodiments of the disclosure;

FIG. 7 is flowchart of methods of performing speech recognitionaccording to embodiments of the disclosure; and

FIG. 8 illustrates an example of performing speech recognition on aspeech signal including utterances spoken by a plurality of speakers,according to an embodiment of the disclosure.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

Throughout the specification, it will be understood that when an elementis referred to as being “connected” or “coupled” to another element, itcan be directly connected to or electrically coupled to the otherelement with one or more intervening elements interposed therebetween.When a part “includes” or “comprises” an element, unless there is aparticular description contrary thereto, the part can further includeother elements, not excluding the other elements.

Throughout the disclosure, the expression “at least one of a, b or c”indicates only a, only b, only c, both a and b, both a and c, both b andc, all of a, b, and c, or variations thereof.

According to the disclosure, functions relate to artificial intelligence(AI) may be performed by a processor and a memory. The processor may becomposed of one or more processors. In this case, the one or moreprocessors may be a general-purpose processor such as a centralprocessing unit (CPU), an application processor (AP), or a digitalsignal processor (DSP), a dedicated graphics processor such as a graphicprocessing unit (GPU) or vision processing unit (VPU), or a dedicated AIprocessor such as a neural processing unit (NPU). The one or moreprocessors may control input data to be processed according topredefined operation rules or an AI model stored in the memory.Alternatively, when the one or more processors are a dedicated AIprocessor, the dedicated AI processor may be designed with a hardwarestructure specialized for processing a specific AI model.

The predefined operation rules or AI model may be created by a trainingprocess. This means that the predefined operation rules or AI model setto perform desired characteristics (or purpose) are created by traininga basic AI model with a learning algorithm that utilizes a large numberof training data. The training process may be performed by a device forperforming AI or a separate server and/or system. Examples of thelearning algorithm may include supervised learning, unsupervisedlearning. semi-supervised learning, and reinforcement learning, butembodiments of the disclosure are not limited thereto.

The AI model may be composed of a plurality of neural network layers.Each of the neural network layers has a plurality of weight values andperforms neural network computations via arithmetic operations onresults of calculations in a previous layer and a plurality of weightvalues in the current layer. A plurality of weights in each of theneural network layers may be optimized by a result of training the AImodel. For example, a plurality of weights may be updated to reduce orminimize a loss or cost value acquired by the AI model during a trainingprocess. An artificial neural network (ANN) may include a deep neuralnetwork (DNN) and may be, for example, a convolutional neural network(CNN), a recurrent neural network (RNN), a restricted Boltzmann machine(RBM), a deep belief network (DBN), a bidirectional recurrent DNN(BRDNN), or a deep Q-network (DQN) but is not limited thereto.

Hereinafter, embodiments of the disclosure will be described in detailwith reference to the accompanying drawings.

FIG. 1 is a network configuration diagram showing a speech recognitionsystem according to an embodiment of the disclosure.

FIG. 1 illustrates a network environment for a speech recognition systemincluding an electronic device 1000, a server 2000, and a network 3000.

A speech recognition function to be described below refers to theability of converting a speech signal including a user's voice into acharacter string (or text). A character string (or text) obtainedthrough conversion from a speech signal by using a speech recognitionfunction may be referred to as a speech recognition result. A userspeech signal may include a voice command, and a speech recognitionresult may also include a command corresponding to the voice command.The voice command may be used to perform a specific function included inthe speech recognition system. The voice command and the text obtainedas a result of speech recognition may be each a sentence, words, orphrases in the form of a natural language.

Furthermore, throughout the specification, the term “registered” meansthat a text or speaker is registered, with the speech recognitionsystem, as a user or information related thereto. A “registered user”refers to a user that has completed user registration with the speechrecognition system. A person may be registered as a user with the speechrecognition system according to the disclosure, and may input his or hervoice for user registration.

The electronic device 1000 is an example of a device for performing AIand may be a speech recognition speaker device provided with a voicecontrol function to perform a specific function.

Furthermore, the electronic device 1000 may be implemented as a devicefor performing a speech recognition function. For example, theelectronic device 1000 may be implemented as various devices such as asmart TV, a set-top box, a mobile phone, a tablet PC, a digital camera,a laptop computer, a desktop computer, an electronic book terminal, adigital broadcasting terminal, a personal digital assistant (PDA), aportable multimedia player (PMP), a wearable device, etc. According toan embodiment of the disclosure, the electronic device 1000 is notlimited to the above examples and may be implemented as various types ofdevices capable of performing speech recognition and speaker recognitionon a speaker's voice.

According to an embodiment of the disclosure, when a speaker's voice isreceived, the electronic device 1000 may recognize both a speaker andspeech spoken by the speaker, extract a command contained in the voiceto perform an operation in response to the command, and output a resultas speech.

Furthermore, according to an embodiment of the disclosure, theelectronic device 1000 may use one or more trained data recognitionmodels to perform speaker recognition and speech recognition on areceived speech signal. For example, the electronic device 1000 may usea data recognition model based on a neural network such as a DNN, a RNN,or the like.

By using a neural network-based data recognition model, the electronicdevice 1000 may produce speech characteristic data by analyzing a speechsignal input via a microphone (1620 of FIG. 2 or 3 ) and perform speechrecognition and speaker recognition based on the speech characteristicdata.

According to an embodiment of the disclosure, the speech characteristicdata may include information generated by analyzing speech signalcharacteristics such as a waveform, a frequency, and an amplitude of thespeech signal.

The electronic device 1000 may perform speech recognition on a firstspeech signal by analyzing speech signal characteristics of the firstspeech signal. For example, the electronic device 1000 may obtain a textcorresponding to the first speech signal by using speech signalcharacteristics corresponding to a prestored text and the analyzedspeech signal characteristics. Based on a result of performing thespeech recognition, the electronic device 1000 may convert the firstspeech signal into a character string (or text). After detecting a textpre-registered by the speaker in the character string, the electronicdevice 1000 may perform speaker recognition on a second speech signalreceived after the first speech signal in order to recognize theidentity of the speaker who pre-registered the text. Otherwise, when thepre-registered text is not detected in the received speech signal, theelectronic device 1000 may not perform speaker recognition on the secondspeech signal received after the first speech signal but extract aspeaker's voice command from the second speech signal to perform anoperation in response to the voice command.

By analyzing speech signal characteristics of the second speech signal,the electronic device 1000 may determine whether the speaker who uttersthe second speech signal corresponds to a pre-registered speaker. Forexample, the electronic device 1000 may determine whether the speakerwho utters the second speech signal corresponds to a pre-registeredspeaker by using speech signal characteristics associated with thepre-registered speaker and speech signal characteristics of the secondspeech signal.

Before performing speech recognition and speaker recognition accordingto an embodiment of the disclosure, the electronic device 1000 may storespeech signal characteristics associated with the pre-registered speakerby pre-registering the speaker talking to the electronic device 1000.For example, during registration of the speaker, the electronic device1000 receives a speaker's voice and analyzes the received speaker'svoice to prestore speech signal characteristics associated with thespeaker. The speech signal characteristics associated with thepre-registered speaker may include information necessary for speakerrecognition, such as a waveform, a frequency, an amplitude, etc. of aspeech signal from the speaker

The electronic device 1000 may obtain and execute a voice command byperforming speech recognition on the second speech signal according to aresult of performing the speaker recognition. For example, theelectronic device 1000 may extract frequency characteristics of a speechsignal and perform speech recognition by using an acoustic model and alanguage model. The electronic device 1000 may convert a speaker's voicein the second speech signal into a character string (or text) andperform natural language processing on the character string (or text) toextract a speaker's voice command included in the second speech signal.A speech recognition result may include the speaker's voice command, andan operation corresponding to the speech recognition result may mean anoperation as a response to the speaker's voice command. The electronicdevice 1000 may extract a command contained in the speaker's voice by aresult of performing the speech recognition and perform operations inresponse to the command according to a result of performing the speakerrecognition.

Examples of the operations that are to be performed by the electronicdevice 1000 in response to the voice command extracted as a result ofperforming the speech recognition may include providing speechinformation, music playback, Internet shopping, financial transaction,call connection, message transmission, alarm setting, and control of anelectronic or mechanical device connected to the electronic device 1000via the network 3000.

For example, when the electronic device 1000 is connected to a smart TVvia the network 3000, the electronic device 1000 may perform operationsincluding channel watching, channel searching, video playing, programsearching, etc. For example, when the electronic device 1000 isconnected to a home appliance such as a smart refrigerator, theelectronic device 1000 may perform operations including checking ofcooling and freezing conditions, temperature setting, etc. However, inthe disclosure, the operations that may be performed by the electronicdevice 1000 are not limited to the above examples.

When performing speaker recognition or an operation in response to avoice command, the electronic device 1000 may generate a syntheticspeech signal for reporting a result of the speaker recognition or aresult of performing the operation. When a first text is detected in thefirst speech signal, the electronic device 1000 may generate a syntheticspeech signal for reporting that speaker recognition is to besubsequently performed for a first speaker. Furthermore, when it isdetermined that the speaker who utters the second speech signal is not aspeaker who registered the first text and thus an operationcorresponding to a speech recognition result is not performed, theelectronic device 1000 may generate a synthetic speech signal forreporting that the operation is not performed.

The electronic device 1000 may communicate with an external device (notshown) via the network 3000 by using a wireless or wired communicationmethod.

Communication methods used by the network 3000 are not limited thereto,and may include not only a communication using a communication network(e.g., a mobile communication network, wired Internet, wirelessInternet, and a broadcast network) but also a short-range wirelesscommunication with the electronic device 1000. For example, the network3000 may include one or more networks including a personal area network(PAN), a local area network (LAN), a campus area network (CAN), ametropolitan area network (MAN), a wide area network (WAN), a broadbandnetwork (BBN), the Internet, etc. The network 3000 may include any oneor more of network topologies including a bus network, a star network, aring network, a mesh network, a star-bus network, a tree or hierarchicalnetwork, etc., but is not limited thereto.

The server 2000 may communicate with the electronic device 1000 via thenetwork 3000 and may be implemented as at least one computer device. Theserver 2000 may be distributed in a cloud form and may provide commands,codes, files, content, etc. to the electronic device 1000.

The server 2000 may convert a speech signal received from the electronicdevice 1000 into a character string (or text) to generate a speechrecognition result. The server 2000 may synthesize a speech to bereproduced in the electronic device 1000 to generate a synthetic speechsignal and transmit the synthetic speech signal to the electronic device1000.

The server 2000 may perform operations that are to be performed by theelectronic device 1000. For example, when the electronic device 1000performs a function of providing speech information, the server 2000 mayrecognize an information request included in a speech signal receivedfrom the electronic device 1000, generate a result corresponding to theinformation request, and transmit the result to the electronic device1000 in the form of a synthetic speech signal. When the electronicdevice 1000 performs a call connection function, the server 2000 mayrecognize a call connection request included in a speech signal receivedfrom the electronic device 1000, perform a call connection in responseto the call connection request, and relay a transmitting signal and areceiving signal during the call connection.

Furthermore, according to an embodiment of the disclosure, the server2000 may perform speech recognition and speaker recognition that may beperformed by the electronic device 1000. For example, the server 2000may receive from the electronic device 1000 a speech signal intervalcorresponding to the first text detected based on a result of performingspeech recognition on the first speech signal. The server 2000 mayperform speaker recognition for recognizing the first speaker whoregistered the first text with respect to the speech signal intervalcorresponding to the first text. The server 2000 may transmit a resultof performing the speaker recognition to the electronic device 1000. Theelectronic device 1000 may perform, based on the speaker recognitionresult received from the server 2000, speaker recognition forrecognizing the first speaker with respect to the second speech signalacquired after the first speech signal and execute a voice commandextracted from the second speech signal according to a result ofperforming the speaker recognition.

The server 2000 may be connected to an external device (not shown) viathe network 3000, and may control the external device according to acontrol command included in a speech signal received from the electronicdevice 1000.

A network for connecting the electronic device 1000 to the server 2000may be of a different type than a network for connecting the externaldevice to the server 2000. For example, the network for connecting theelectronic device 1000 to the server 2000 may be a local area network(LAN) or the Internet, and the network for connecting the externaldevice to the server 2000 may be a mobile communication network.

The external device may be a device that is controllable in response toa voice command received from the electronic device 1000. For example,the external device may be a mobile phone, smartphone, tablet or laptopcomputer, which may be hand-held by the user, and a home appliance suchas a smart TV, personal computer (PC), refrigerator, or washing machine.

Although FIG. 1 shows that the electronic device 1000 is connected tothe server 2000 for performing a speech recognition function via thenetwork 3000, this is merely an example, and the electronic device 1000may independently perform a speech recognition function.

FIG. 2 is a block diagram of an internal configuration of an electronicdevice 1000 according to an embodiment of the disclosure, and FIG. 3 isa detailed block diagram of an internal configuration of an electronicdevice 1000 according to an embodiment of the disclosure.

Referring to FIG. 2 , the electronic device 1000 may include a processor1300 and a microphone 1620. However, all the components shown in FIG. 2are not essential components of the electronic device 1000. Theelectronic device 1000 may include more or fewer components than thoseshown in FIG. 2 .

For example, referring to FIG. 3 , according to some embodiments of thedisclosure, the electronic device 1000 may include a camera 1610, amemory 1700, a display 1210, and a processor 1300. The electronic device1000 may further include a user inputter 1100, a communicator 1500(e.g., a transceiver), an outputter 1200, a sensing unit 1400, and anaudio/video (A/V) inputter 1600.

The user inputter 1100 is a device via which the user inputs datanecessary for controlling the electronic device 1000. Examples of theuser inputter 1100 may include, but are not limited to, a keypad, a domeswitch, a touch pad (a capacitive overlay type, a resistive overlaytype, an infrared beam type, a surface acoustic wave type, an integralstrain gauge type, a piezoelectric type, etc.), a jog wheel, and a jogswitch.

According to an embodiment of the disclosure, the user inputter 1100 mayreceive a user input for performing speech recognition.

The outputter 1200 may output an audio signal, a video signal, or avibration signal. The outputter 1200 may include a display 1210, anaudio outputter 1220, and a vibration motor 1230.

The display 1210 may display and output information processed by theelectronic device 1000. According to an embodiment of the disclosure,the display 1210 may output results of performing speaker recognitionand speech recognition.

In addition, when the display 1210 and a touch pad form a layerstructure to form a touch screen, the display 1210 may be used as aninput device as well as an output device. The display 1210 may includeat least one of a liquid crystal display (LCD), athin-film-transistor-LCD (TFT-LCD), an organic light-emitting diode(OLED), a flexible display, a three-dimensional (3D) display, or anelectrophoretic display. Furthermore, the electronic device 1000 mayinclude two or more displays 1210 according to its implementedconfiguration.

The audio outputter 1220 may output audio data received from thecommunicator 1500 or stored in the memory 1700. According to anembodiment of the disclosure, the audio outputter 1220 may output as asynthetic speech sound a character string representing the results ofperforming the speaker recognition and speech recognition. For example,when a preset text is detected as a speech recognition result, the audiooutputter 1220 may output a character string indicating that speakerrecognition is to be performed as a synthetic speech sound. Furthermore,after the speech recognition and the speaker recognition are performed,the audio outputter 1220 may output a character string indicating aresult of executing a voice command as a synthetic speech sound.

The vibration motor 1230 may output a vibration signal. Furthermore, thevibration motor 1230 may output a vibration signal when a touch is inputon a touch screen. According to an embodiment of the disclosure, thevibration motor 1230 may output a vibration signal for indicating thatspeech recognition and speaker recognition are to be performed.

The processor 1300 controls all operations of the electronic device1000. For example, the processor 1300 may control the user inputter1100, the outputter 1200, the sensing unit 1400, the communicator 1500,and the A/V inputter 1600 in its entirety by executing programs storedin the memory 1700. The electronic device 1000 may include at least oneprocessor 1300.

The processor 1300 may be configured to process instructions of acomputer program by performing basic arithmetic, logic, and input/outputoperations. A command may be provided from the memory 1700 to theprocessor 1300 or may be received via the communicator 1500 and providedto the processor 1300. For example, the processor 1300 may be configuredto execute instructions according to program code stored in a recordingdevice such as the memory 1700.

According to an embodiment of the disclosure, the processor 1300 mayacquire first and second speech signals corresponding to a speaker'svoice based on audio signals produced by the microphone 1620. The secondspeech signal may be acquired after the first speech signal. Theprocessor 1300 may perform speaker recognition on the second speechsignal, when a first text preset for performing speaker recognition isdetected by a result of performing speech recognition on the firstspeech signal. According to an embodiment of the disclosure, theprocessor 1300 may perform speaker recognition on the second speechsignal only when a result of performing speaker recognition on thespeech signal interval indicates that a speaker of a speech signalinterval in which a first text is uttered corresponds to a first speakerwho registered the first text. When a result of performing the speakerrecognition on the second speech signal indicates that a speaker of thesecond speech signal corresponds to the first speaker, the processor1300 may execute a voice command extracted from the second speechsignal.

Otherwise, when a result of performing the speaker recognition on thesecond speech signal indicates that the speaker of the second speechsignal does not correspond to the first speaker, the processor 1300 mayignore the voice command extracted from the second speech signal.

Furthermore, the processor 1300 may detect a plurality of preset textsin a speech signal and perform speaker recognition for a plurality ofspeakers respectively corresponding to the preset texts. The processor1300 may execute, based on the order of priority for the speakers, avoice command extracted from a speech signal corresponding to eachspeaker. For example, when a second text is detected by performingspeech recognition on a third speech signal acquired after the firstspeech signal, the processor 1300 may perform speaker recognition on afourth speech signal acquired after the third speech signal. When aresult of the speaker recognition indicates that a speaker of the fourthspeech signal corresponds to a second speaker who registered the secondtext, the processor 1300 may determine the order of priority for thesecond speaker and the first speaker who registered the first textextracted from the first speech signal. When the first speaker has ahigher priority than the second speaker, the processor 1300 may notexecute a voice command extracted from the fourth speech signal.Otherwise, when the second speaker has a higher priority than the firstspeaker, the processor 1300 may execute the voice command extracted fromthe fourth speech signal.

The sensing unit 1400 may detect a status of the electronic device 1000or the surroundings thereof and transmit information about the detectedstatus to the processor 1300.

The sensing unit 1400 may include at least one of a magnetic sensor1410, an acceleration sensor 1420, a temperature/humidity sensor 1430,an infrared sensor 1440, a gyroscope sensor 1450, a position sensor(e.g., GPS) 1460, a barometric pressure sensor 1470, a proximity sensor1480, or an RGB (red, green, and blue) sensor (an illuminance sensor)1490, but is not limited thereto.

The communicator 1500 may include one or more components that enable theelectronic device 1000 to communicate with the server (2000 of FIG. 1 )or an external device (not shown). For example, the communicator 1500may include a short-range wireless communicator 1510, a mobilecommunicator 1520 (e.g., a mobile transceiver), and a broadcast receiver1530.

The short-range wireless communicator 1510 (e.g., a short-rangetransceiver) may include a Bluetooth communication module, a BluetoothLow Energy (BLE) communication module, a Near Field Communication (NFC)module, a wireless local area network (WLAN) or Wi-Fi communicationmodule, a Zigbee communication module, an Infrared Data Association(IrDA) communication module, a Wi-Fi Direct (WFD) communication module,an Ultra-wideband (UWB) communication module, and an Ant+ communicationmodule, but is not limited thereto.

The mobile communicator 1520 transmits or receives a wireless signal toor from at least one of a base station, an external terminal, or aserver on a mobile communication network. In this case, the wirelesssignal may be a voice call signal, a video call signal, or data in anyone of various formats according to transmission and reception of atext/multimedia message.

The broadcast receiver 1530 receives broadcast signals and/orbroadcast-related information from the outside via a broadcast channel.The broadcast channel may include a satellite channel and a terrestrialchannel According to an embodiment of the disclosure, the electronicdevice 1000 may not include the broadcast receiver 1530.

According to an embodiment of the disclosure, the communicator 1500 mayreceive results of speech recognition and speaker recognition, which aretransmitted by the server 2000, or receive a speech signal transmittedby the external device.

The A/V inputter 1600 for inputting an audio or video signal may includethe camera 1610, the microphone 1620, etc. The camera 1610 may acquirean image frame such as a still or moving image via an image sensor in avideo call mode or capture mode. An image captured through the imagesensor may be processed by the processor 1300 or a separate imageprocessor (not shown).

The microphone 1620 may receive an external acoustic signal and processthe acoustic signal as electrical speech data. For example, themicrophone 1620 may receive a speaker's speech signal. As anotherexample, the microphone 1620 may receive a speech signal includingutterances by a plurality of speakers.

The memory 1700 may store programs necessary for processing or controloperations performed by the processor 1300 or store data input to oroutput from the electronic device 1000. According to an embodiment ofthe disclosure, the memory 1700 may store information necessary forspeaker recognition, such as information necessary for recognizing apre-registered speaker from a speech signal. Furthermore, the memory1700 may store information necessary for speech recognition, such as anacoustic model, a language model, etc.

According to an embodiment of the disclosure, when speaker recognitionand speech recognition are performed by the server 2000, informationnecessary for the speaker recognition and the speech recognition may bestored in the server 2000 instead of the memory 1700.

The memory 1700 may include at least one type of storage medium, forexample, a flash memory-type memory, a hard disk-type memory, amultimedia card micro-type memory, a card-type memory (e.g., an SD cardor an XD memory), random access memory (RAM), static RAM (SRAM),read-only memory (ROM), electrically erasable programmable ROM (EEPROM),PROM, a magnetic memory, a magnetic disc, or an optical disc.

Programs stored in the memory 1700 may be classified into a plurality ofmodules according to their functions. For example, the programs may beclassified into a user interface (UI) module 1710, a touch screen module1720, and a notification module 1730.

The UI module 1710 may provide a specialized UI, a GUI, etc.interworking with the electronic device 1000 for each application. Thetouch screen module 1720 may detect a user's touch gesture on a touchscreen and transmit information about the detected touch gesture to theprocessor 1300. According to some embodiments of the disclosure, thetouch screen module 1720 may recognize and analyze a touch code. Thetouch screen module 1720 may be formed by separate hardware componentsincluding a controller.

Various sensors may be positioned within or near the touch screen so asto sense a touch or proximity touch on the touch screen. A tactilesensor is an example of the sensors for sensing a touch on the touchscreen. The tactile sensor is used to sense a touch of a particularobject to the same or greater degree than the degree to which a humancan sense the touch. The tactile sensor may detect various pieces ofinformation including the roughness of a contact surface, rigidity of acontact object, and the temperature of a contact point.

A user's touch gesture may include tap, touch and hold, double-tap,drag, panning, flick, drag and drop, swipe, etc.

The notification module 1730 may generate a signal for notifying theoccurrence of an event in the electronic device 1000.

FIG. 4 illustrates an example of performing speech recognition accordingto an embodiment of the disclosure.

According to an embodiment of the disclosure, an electronic device 1000for performing speech recognition may be installed fixedly in a specificspace to simultaneously receive speeches uttered by a plurality ofspeakers.

Reference element 401 of FIG. 4 shows an example in which speechrecognition is performed without speaker recognition, according to anembodiment of the disclosure, and reference element 402 shows an examplein which speech recognition is performed based on speaker recognition,according to an embodiment of the disclosure.

Referring to reference element 401 of FIG. 4 , the electronic device1000 for performing speech recognition according to an embodiment of thedisclosure may perform speech recognition by receiving utterances by thefirst speaker 410 and the second speaker 420 For example, the electronicdevice 1000 may perform speech recognition on a first utterance 411 bythe first speaker 410 and a second utterance 421 by the second speaker420. As a result of performing the speech recognition, the electronicdevice 1000 may respectively obtain “Hi Speaker, mute a sound” and “HiSpeaker, play a song” as texts respectively corresponding to the firstand second utterances 411 and 421.

As the first and second speakers 410 and 420 respectively utter voicecommands, each including the hotword “Hi Speaker”, the electronic device1000 may perform speech recognition on speech uttered after the hotword“Hi Speaker”. A hotword refers to a specific word or phrase that may bespoken by a user to invoke the electronic device 1000 before a user'sutterance of a voice command.

After recognizing “Hi Speaker” that is a predefined hotword from thefirst utterance 411 by performing speech recognition, the electronicdevice 1000 may obtain “mute a sound” that is speech uttered after “HiSpeaker” as a voice command. Similarly, the electronic device 1000 mayobtain “play a song” as a voice command after recognizing “Hi Speaker”from the second utterance 421 via speech recognition. The electronicdevice 1000 may perform operations in response to the obtained voicecommands.

However, according to the example shown in 401 of FIG. 4 , when thewords “mute a sound” and “play a song” respectively spoken by the firstand second speakers 410 and 420 are both obtained as voice commands, thewords “mute a sound” and “play a song” that are contradictory voicecommands may be processed together.

On the other hand, according to the example shown in reference element402 of FIG. 4 , as speaker recognition is performed, the voice commanduttered by a third speaker 430 who is a pre-registered speaker may beprocessed preferentially to the voice command uttered by a fourthspeaker 440.

Referring to reference element 402 of FIG. 4 , the electronic device1000 for performing speech recognition according to an embodiment of thedisclosure may perform speech recognition by receiving third and fourthutterances 431 and 441 respectively spoken by the third and fourthspeakers 430 and 440. For example, the electronic device 1000 mayperform speech recognition on the third utterance 431 by the thirdspeaker 430 and the fourth utterance 441 by the fourth speaker 440. As aresult of performing the speech recognition, the electronic device 1000may respectively obtain “Hi Speaker, it's mom, mute a sound” and “HiSpeaker, play a song” as texts respectively corresponding to the thirdand fourth utterances 431 and 441. According to an embodiment of thedisclosure, after receiving an utterance of “it's mom” included in thethird utterance 431, the electronic device 1000 may receive thefollowing words “mute a sound” in the third utterance 431 and “HiSpeaker, play a song” in the fourth utterance 441.

After “it's mom” that is a pre-registered text is recognized in thethird utterance 431 by the third speaker 430 as a result of performingthe speech recognition, the electronic device 1000 may perform bothspeech recognition and speaker recognition on the subsequently receivedutterances “mute a sound” and “Hi Speaker, play a song”. Thus, theelectronic device 1000 may process “mute a sound” that is a voicecommand uttered by the third speaker 430 who registered “it's mom”preferentially to “Hi Speaker, play a song” that is a voice commanduttered by the fourth speaker 440.

According to an embodiment of the disclosure, the electronic device 1000may perform only speech recognition on an input speech signal withoutperforming speaker recognition and then recognize the pre-registeredtext “it's mom” as a result of the speech recognition. The electronicdevice 1000 may perform both speaker recognition and speech recognitionon a speech signal received during a preset time interval afterrecognizing “it's mom”. The electronic device 1000 may determine whetherthe received speech signal is an utterance by the third speaker 430 whoregistered “it's mom”. As a result of the speaker recognition, theelectronic device 1000 may process “mute a sound” that is the voicecommand uttered by the third speaker 430. Furthermore, as a result ofthe speaker recognition, the electronic device 1000 may not process“play a song” that is the voice command uttered by the fourth speaker430.

FIG. 5 illustrates an example of performing speech recognition accordingto an embodiment of the disclosure.

Referring to FIG. 5 , the electronic device 1000 may receive a speechsignal including an utterance 510 by a first speaker and an utterance520 by a second speaker. In FIG. 5 , an interval indicated by arrowsrepresents a speech signal interval during which the first or secondspeaker utters a speech signal.

Speech signals of FIG. 5 may include the utterances 510 and 520respectively spoken by the first and second speakers. For example, afirst speech signal may include the utterances 510 and 520 respectivelyspoken by the first and second speaker in a first interval 530.Furthermore, a second speech signal may include the utterances 510 and520 respectively spoken by the first and second speaker in a secondinterval 540.

The electronic device 1000 may perform speech recognition on the firstspeech signal acquired in the first interval 530. Third and fourth texts512 and 521 respectively corresponding to the utterances 510 and 520 bythe first and second speakers may be obtained as a result of performingthe speech recognition on the first speech signal acquired in the firstinterval 530. After a first text 511 that is a pre-registered text isdetected from the third text 512, the electronic device 1000 may performboth speech recognition and speaker recognition on the second speechsignal acquired in the second interval 540.

For example, after the first text 511 is detected, speaker recognitionmay be performed for the first speaker who registered the first text511. After the first text 511 is detected, the electronic device 1000may acquire information necessary to perform speaker recognition for thefirst speaker.

Furthermore, when the first text 511 that is the pre-registered text isdetected in the third text 512, the electronic device 1000 may performspeaker recognition for recognizing the first speaker on a speech signalinterval in which the first text 511 is uttered. According to a resultof the speaker recognition on the speech signal interval, the electronicdevice 1000 may determine whether to perform, together with speechrecognition, the speaker recognition for recognizing the first speakerwith respect to a speech signal acquired in the second interval 540.

When it is determined, as a result of the speaker recognition, that thefirst text 511 is uttered by the first speaker, the electronic device1000 may perform both speech recognition and speaker recognition on thesecond speech signal acquired in the second interval 540. On the otherhand, when it is determined that the first text 511 is not uttered bythe first speaker, the electronic device 1000 may perform only speechrecognition on the second speech signal acquired in the second interval540 without performing speaker recognition thereon.

The second interval 540 during which both speaker recognition for thefirst speaker and speech recognition are performed may be set as apreset time interval following acquisition of a speech signal in thefirst interval 530. The second interval 540 is not limited to the aboveexample and may be set in various ways.

When it is determined that both speech recognition and speakerrecognition are performed on the second speech signal acquired in thesecond interval 540, the electronic device 1000 may perform both speechrecognition and speaker recognition on the utterances 510 and 520 by thefirst and second speakers, which are acquired during the second interval540. A fifth text 513 and a sixth text 522 may be obtained by a resultof performing the speech recognition. The electronic device 1000 maydetermine that the fifth text 513 is uttered by the first speaker byperforming the speaker recognition. Thus, the electronic device 1000 mayextract a voice command from the first text 513 and perform an operationin response to the extracted voice command. Furthermore, because it isdetermined, as a result of performing the speaker recognition, that thesixth text 522 is not uttered by the first speaker, the electronicdevice 1000 may ignore a voice command corresponding to the sixth text522 without executing the voice command.

FIG. 6 is a flowchart of a method of performing speech recognitionaccording to an embodiment of the disclosure.

Referring to FIG. 6 , the electronic device 1000 may perform speechrecognition on a first speech signal (operation 601). The electronicdevice 1000 may obtain a text corresponding to the first speech signalas a result of the speech recognition.

As a result of performing the speech recognition in operation 601, theelectronic device 1000 may detect a first text preset for performingspeaker recognition in a text corresponding to the first speech signal(operation 602).

The first text may be pre-registered by a first speaker with theelectronic device 1000 in order to perform speaker recognition. Forexample, the first text may include a word or phrase indicating a firstspeaker's identity, name, nickname, etc.

By receiving a speech that is an utterance of the first text from thefirst speaker, the electronic device 1000 may set the first text as atext for speaker recognition.

The electronic device 1000 may detect the first text by performing namedentity recognition on a text obtained by performing speech recognitionon the first speech signal. Named entity recognition refers to atechnique for recognizing a named entity corresponding to a predefinedperson, company, location, time expression, unit, or the like in acertain text and adding a tag to the recognized named entity. Forexample, the electronic device 1000 may recognize a named entitycorresponding to a predefined person in a text and add a tagrepresenting the predefined person to the recognized named entity.

According to an embodiment of the disclosure, by performing named entityrecognition, the electronic device 1000 may extract a named entityrepresenting the pre-registered first speaker from a text correspondingto the first speech signal. The named entity representing the firstspeaker, which is to be extracted via named entity recognition, may bepre-trained using AI technology.

According to an embodiment of the disclosure, the electronic device 1000may use trained one or more data recognition models to perform namedentity recognition on a text obtained as a speech recognition result.

According to an embodiment of the disclosure, the electronic device 1000may use a data recognition model based on a neural network such as aDNN, a RNN, etc.

The electronic device 1000 may perform named entity recognition on atext obtained as a speech recognition result by using one or more datarecognition models based on a neural network. For example, theelectronic device 1000 may use a data recognition model to determinewhether a named entity representing the pre-registered first speaker iscontained in the text for speaker recognition.

When the named entity representing the pre-registered first speaker isextracted from the text, the named entity may be detected as the firsttext.

Otherwise, when the preset first text is not detected as a result ofperforming the speech recognition, the electronic device 1000 mayperform only speech recognition on a subsequently acquired speech signallike in operation 601.

When the first text is detected in operation 602, the electronic device1000 may perform speaker recognition on a second speech signal acquiredafter the first speech signal (operation 603). According to anembodiment of the disclosure, the second speech signal undergoing thespeech recognition may include a speech signal acquired during a presettime period after acquisition of the first speech signal.

When a result of performing the speaker recognition in operation 603indicates that a speaker of the second speech signal corresponds to afirst speaker who registered the first text, the electronic device 1000may execute a voice command extracted from the second speech signal(operation 604). Otherwise, when the speaker of the second speech signaldoes not correspond to the first speaker, the electronic device 1000 mayignore the voice command extracted from the second speech signal.

Furthermore, the electronic device 1000 may detect a plurality of presettexts in a speech signal and perform speaker recognition on a pluralityof speakers respectively corresponding to the preset texts. Theelectronic device 1000 may execute, based on the order of priority forthe speakers, a voice command extracted from a speech signalcorresponding to each speaker. For example, when a second text isdetected by performing speech recognition on a third speech signalacquired after the first speech signal, speaker recognition may beperformed on a fourth speech signal acquired after the third speechsignal. When a result of the speaker recognition indicates that aspeaker of the fourth speech signal corresponds to a second speaker whoregistered the second text, the electronic device 1000 may determine theorder of priority for the second speaker and the first speaker whoregistered the first text extracted from the first speech signal. Whenthe first speaker has a higher priority than the second speaker, theelectronic device 1000 may not execute a voice command extracted fromthe fourth speech signal. Otherwise, when the second speaker has ahigher priority than the first speaker, the electronic device 1000 mayexecute the voice command extracted from the fourth speech signal.

FIG. 7 is a flowchart of a method of performing speech recognitionaccording to an embodiment of the disclosure.

Operations 701, 702, 707 of FIG. 7 may respectively correspond to 601,602, and 603 of FIG. 6 .

Referring to FIG. 7 , the electronic device 1000 may perform speechrecognition on a first speech signal (operation 701). The electronicdevice 1000 may obtain a text corresponding to the first speech signalas a result of the speech recognition.

As a result of performing the speech recognition in operation 701, theelectronic device 1000 may detect a first text preset for performingspeaker recognition in a text corresponding to the first speech signal(operation 702). The first text may be pre-registered by a firstspeaker. Furthermore, the first text may be detected via named entityrecognition.

The electronic device 1000 may acquire information necessary to performspeaker recognition for a first speaker who registered the first text(operation 703). The information necessary to perform the speakerrecognition for the first speaker may include information collected whenthe first speaker registers the first text. Furthermore, the informationnecessary to perform the speaker recognition for the first speaker mayinclude information pre-trained based on speech information of the firstspeaker by using AI technology.

The electronic device 1000 may perform speaker recognition on a speechsignal corresponding to the first text (operation 704). For example, theelectronic device 1000 may acquire a speech signal interval in which thefirst text is uttered based on the first speech signal and performspeaker recognition on the speech signal interval.

The electronic device 1000 may determine whether a speaker uttering thefirst text corresponds to the first speaker as a result of performingthe speaker recognition in operation 704 (operation 705). The electronicdevice 1000 may perform speaker recognition by using the informationnecessary to perform speaker recognition for the registered firstspeaker.

For example, the electronic device 1000 may extract feature informationregarding speech in the speech signal interval in which the first textis uttered. The electronic device 1000 may perform speaker recognitionby comparing the extracted feature information with prestored featureinformation regarding the first speaker. When the degree ofcorrespondence between the feature information regarding the speechsignal interval during which the first text is uttered and the featureinformation regarding the first speaker is greater than or equal to afirst reference value, the electronic device 1000 may determine that thespeaker uttering the first text corresponds to the first speaker.

When it is determined, as a result of performing the speakerrecognition, that the speaker uttering the first text corresponds to thefirst speaker, the electronic device 1000 may output informationindicating that speaker recognition is to be performed (operation 706).For example, the electronic device 1000 may output via a speaker asynthetic speech sound indicating that the speaker recognition is to beperformed. A speaker of the second speech signal acquired after thefirst speech signal may recognize, based on the information output bythe electronic device 1000, that the speaker recognition is to beperformed.

Otherwise, when it is determined, as a result of performing the speakerrecognition, that the speaker uttering the first text does notcorrespond to the first speaker, the electronic device 1000 may performspeech recognition on a speech signal acquired after the first speechsignal without performing speaker recognition thereon.

The electronic device 1000 may perform speaker recognition on a secondspeech signal acquired after the first speech signal (operation 707).For example, the electronic device 1000 may extract feature informationregarding speech in the second speech signal. The electronic device 1000may perform speaker recognition by comparing the extracted featureinformation with prestored feature information regarding the firstspeaker. When the degree of correspondence between the featureinformation regarding the speech in the second speech signal and thefeature information regarding the first speaker is greater than or equalto a second reference value, the electronic device 1000 may determinethat the speaker of the second speech signal corresponds to the firstspeaker.

According to an embodiment of the disclosure, when it is determined thatspeaker recognition is performed for recognizing a speaker specified bydetection of the first text, the speaker recognition is performed onlyfor the specified speaker. Thus, the electronic device 1000 is able toperform speaker recognition with high accuracy.

The first reference value used in the speaker recognition in operation704 and the second reference value used in the speaker recognition inoperation 707 may be preset values and may be both set to differentvalues. For example, the first reference value may be set to be lessthan the second reference value. Thus, the speaker recognition inoperation 707 may be performed with higher accuracy than the speakerrecognition in operation 704. The electronic device 1000 may use agreater reference value for the speaker recognition in operation 707than for the speaker recognition in operation 704

When a result of performing the speaker recognition on the second speechsignal indicates that the speaker of the second speech signalcorresponds to the first speaker who registered the first text inoperation 708, the electronic device 1000 may execute a voice commandextracted from the second speech signal (operation 709). For example,the electronic device 1000 may extract a text by a result of performingspeech recognition on the second speech signal and obtain a voicecommand that matches a user's intent based on the extracted text.

Otherwise, when a result of performing the speaker recognition on thesecond speech signal indicates that the speaker of the second speechsignal does not correspond to the first speaker who registered the firsttext in operation 708, the electronic device 1000 may ignore a voicecommand extracted from the second speech signal without executing thevoice command (operation 710). Alternatively, when a result ofperforming the speaker recognition indicates that the speaker of thesecond speech signal does not correspond to the first speaker whoregistered the first text, the electronic device 1000 may ignore and notperform speech recognition on the second speech signal.

According to an embodiment of the disclosure, when the voice command inthe second speech signal is ignored as a result of performing thespeaker recognition, the electronic device 1000 may count the time fordetermine whether to continuously perform speaker recognition for thefirst speaker. When a speech signal determined as an utterance by thefirst speaker is not acquired within a preset time period, as a resultof performing the speaker recognition, the electronic device 1000 maynot perform speaker recognition for the first speaker with respect to asubsequently acquired speech signal. The electronic device 1000 mayperform operations following the operation 701 on the subsequentlyacquired speech signal.

The electronic device 1000 may output information indicating that thevoice command in the second speech signal is ignored (operation 711).For example, the electronic device 1000 may output via a speaker asynthetic speech sound indicating that the voice command in the secondspeech signal is ignored. The speaker of the second speech signalacquired after the first speech signal may recognize, based on theinformation output by the electronic device 1000, that the voice commanduttered by the speaker himself/herself is ignored.

FIG. 8 illustrates an example of performing speech recognition on aspeech signal including utterances spoken by a plurality of speakers,according to an embodiment of the disclosure.

Referring to FIG. 8 , the electronic device 1000 may receive a speechsignal including an utterance 810 by a first speaker and an utterance820 by a second speaker. An interval indicated by arrows represents aspeech signal interval during which the first or second speaker utters aspeech signal.

A speech signal acquired in each interval may include the utterances 810and 820 respectively spoken by the first and second speakers.

The electronic device 1000 may perform speech recognition on speechsignals acquired in a first interval 830. Third and fourth texts 812 and821 respectively corresponding to the utterances 810 and 820 by thefirst and second speakers may be obtained as a result of performing thespeech recognition on the first speech signal acquired in the firstinterval 830. After a first text 811 that is a pre-registered text isdetected in the third text 812, the electronic device 1000 may performboth speech recognition and speaker recognition for the first speakerwith respect to a second speech signal acquired in the second interval840.

For example, after the first text 811 is detected, speaker recognitionmay be performed for the first speaker who registered the first text811. After the first text 811 is detected, the electronic device 1000may acquire information necessary to perform speaker recognition for thefirst speaker.

Furthermore, when the first text 811 that is the pre-registered text isdetected in the third text 812, the electronic device 1000 may performspeaker recognition on a speech signal interval in which the first text511 is uttered. According to a result of the speaker recognition on thespeech signal interval, the electronic device 1000 may determine whetherto perform, together with speech recognition, the speaker recognitionfor recognizing the first speaker with respect to speech signalsacquired in the second interval 840.

When it is determined, as a result of the speaker recognition, that thefirst text 811 is uttered by the first speaker, the electronic device1000 may perform both speech recognition and speaker recognition on thesecond speech signal acquired in the second interval 840. On the otherhand, when it is determined that the first text 811 is not uttered bythe first speaker, the electronic device 1000 may perform only speechrecognition on the second speech signal acquired in the second interval840 without performing speaker recognition thereon.

The second interval 840 during which both speaker recognition for thefirst speaker and speech recognition are performed may be set as apreset time interval following acquisition of speech signals in thefirst interval 830.

When it is determined that both speech recognition and speakerrecognition are performed on the second speech signal acquired in thesecond interval 840, the electronic device 1000 may perform both speechrecognition and speaker recognition on utterances 810 and 820 by thefirst and second speakers, which are acquired during the second interval840. A fifth text 813 and a sixth text 822 may be obtained by a resultof performing the speech recognition. The electronic device 1000 maydetermine that the fifth text 813 is uttered by the first speaker byperforming the speaker recognition. Thus, the electronic device 1000 mayextract a voice command from the first text 813 and perform an operationin response to the extracted voice command. Furthermore, because it isdetermined that the sixth text 822 is not uttered by the first speakeras a result of performing the speaker recognition, the electronic device1000 may ignore a voice command corresponding to the sixth text 822without executing the voice command.

The electronic device 1000 may detect in the sixth text 822 a secondtext 823 preset for performing speaker recognition for the secondspeaker. The second text 823 may be pre-registered by the secondspeaker. After the second text 823 is detected, the electronic device1000 may perform both speech recognition and speaker recognition for thesecond speaker with respect to a speech signal subsequently acquired ina third interval 850.

For example, after the second text 823 is detected, speaker recognitionmay be performed for the second speaker who registered the second text823. After the second text 823 is detected, the electronic device 1000may acquire information necessary to perform speaker recognition for thesecond speaker.

Furthermore, when the second text 823 that is preset for performing thespeaker recognition for the second speaker is detected in the sixth text822, the electronic device 1000 may perform speaker recognition for thesecond speaker on a speech signal interval in which the second text 823is uttered. According to a result of the speaker recognition on thespeech signal interval, the electronic device 1000 may determine whetherto perform, together with speech recognition, the speaker recognitionfor the second speaker with respect to the speech signal acquired in thethird interval 850.

When it is determined, as a result of the speaker recognition, that thesecond text 823 is uttered by the second speaker, the electronic device1000 may perform both speech recognition and speaker recognition on thespeech signal acquired in the third interval 850. On the other hand,when it is determined that the second text 823 is not uttered by thesecond speaker, the electronic device 1000 may perform only speechrecognition on the speech signal acquired in the third interval 850without performing speaker recognition thereon.

The second interval 850 during which both speaker recognition for thefirst speaker and speech recognition are performed may be set as apreset time interval following acquisition of a speech signal in thefirst interval 830. Furthermore, the third interval 850 during whichboth speaker recognition for the second speaker and speech recognitionare performed may be set as a preset time interval following acquisitionof a speech signal corresponding to the second text 823. The second andthird intervals 840 and 850 are not limited to the above examples andmay be set in various ways.

When it is determined that both speech recognition and speakerrecognition for the second speaker are performed on the speech signalacquired in the third interval 850, the electronic device 1000 mayperform both speech recognition and speaker recognition on theutterances 810 and 820 by the first and second speakers, which areacquired during the third interval 850. Furthermore, the electronicdevice 1000 may perform speech recognition, speaker recognition for thefirst speaker, and speaker recognition for the second speaker on aspeech signal acquired in a fourth interval 860 in which the second andthird intervals 840 and 850 overlap each other.

As a result of performing the speech recognition, seventh and eighthtexts 814 and 824 may be obtained in the fourth interval 860. Byperforming the speaker recognition for the first and second speakers,the electronic device 1000 may determine that seventh and eighth texts814 and 824 are uttered by the first and second speakers. The electronicdevice 1000 may determine the order of priority for the first and secondspeakers and perform, according to the determined order of priority, anoperation in response to a voice command corresponding to one of theseventh and eighth texts 814 and 824. For example, when the firstspeaker has a higher priority than the second speaker, the electronicdevice 1000 may preferentially perform a voice command corresponding tothe seventh text 814 uttered by the first speaker. Furthermore, theelectronic device 1000 may not execute a voice command corresponding tothe eighth text 824 uttered by the second speaker.

The electronic device 1000 may perform both speaker recognition for thesecond speaker and speech recognition during an interval between T1 andT2 in the third interval 850 The electronic device 1000 may not performspeaker recognition for the first speaker. Ninth and tenth texts 815 and825 may be obtained as a speech recognition result. The electronicdevice 1000 may determine that the tenth text 825 is uttered by thesecond speaker by performing speaker recognition for the second speaker.Thus, the electronic device 1000 may extract a voice command from thetenth text 825 and perform an operation in response to the extractedvoice command. Furthermore, because it is determined that the ninth text815 is not uttered by the second speaker, the electronic device 1000 mayignore a voice command corresponding to the ninth text 815 withoutexecuting the voice command.

According to an embodiment of the disclosure, speaker recognition may beperformed on utterances spoken after detection of a preset text and noton all utterances during speech recognition, thereby reducing the amountof computation.

Furthermore, according to an embodiment of the disclosure, speakerrecognition may be performed for a speaker who registered a preset textafter detection of the preset text and not for all speakers, therebyincreasing accuracy of the speaker recognition.

Embodiments may be implemented through computer-readable recording mediahaving recorded thereon computer-executable instructions such as programmodules that are executed by a computer. The computer-readable recordingmedia may be any available media that can be accessed by a computer andinclude both volatile and nonvolatile media and both detachable andnon-detachable media. Furthermore, the computer-readable recording mediamay include computer storage media and communication media. The computerstorage media include both volatile and nonvolatile and both detachableand non-detachable media implemented by any method or technique forstoring information such as computer-readable instructions, datastructures, program modules, or other data. The communication mediatypically embody computer-readable instructions, data structures, orprogram modules, and include any information transmission media.

Furthermore, in the specification, the term “unit” may be a hardwarecomponent such as a processor or circuit and/or a software componentthat is executed by a hardware component such as a processor.

The above description of the disclosure is provided for illustration,and it will be understood by those of ordinary skill in the art thatvarious changes in form and details may be readily made therein withoutdeparting from essential features and the spirit and scope of thedisclosure as defined by the following claims. Accordingly, the aboveembodiments of the disclosure and all aspects thereof are examples onlyand are not limiting. For example, each component defined as anintegrated component may be implemented in a distributed fashion.Likewise, components defined as separate components may be implementedin an integrated manner.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A method of performing speech recognition by anelectronic device, the method comprising: registering a user by storingcharacteristic information of voice signals of the user as registrationinformation of the user for executing at least one voice command;receiving a speech signal including a first text and a voice commanduttered subsequently to the first text; determining whether the firsttext is detected in the received speech signal; determiningcharacteristic information of the received speech signal, based ondetermining that the first text is detected in the speech signal;determining whether a speaker who uttered the first text and the voicecommand corresponds to the registered user by comparing thecharacteristic information of the received speech signal with thecharacteristic information stored as the registration information of theregistered user; executing the voice command and outputting a result ofexecuting the voice command, based on determining that the speaker whouttered the first text and the voice command corresponds to theregistered user; and ignoring the voice command, based on determiningthat the speaker who uttered the first text and the voice command doesnot correspond to the registered user.
 2. The method of claim 1, whereinthe at least one voice command includes at least one of Internetshopping, financial transaction, call connection, message transmissionand alarm setting.
 3. The method of claim 1, wherein the outputting ofthe result of executing the voice command comprises: outputting asynthetic speech signal generated by executing the voice command; anddisplaying an image generated by executing the voice command.
 4. Anelectronic device for performing speech recognition, the electronicdevice comprising: a microphone; at least one memory storing one or moreinstructions; and at least one processor configured to, by executing theone or more instructions, to: control the memory to register a user bystore characteristic information of voice signals of the user asregistration information of the user for executing at least one voicecommand, receive, through the microphone, a speech signal including afirst text and a voice command uttered subsequently to the first text,determine whether a first text is detected in the received speechsignal, determine characteristic information of the received speechsignal, based on determining that the first text is detected in thespeech signal, determine whether a speaker who uttered the first textand the voice command corresponds to the registered user by comparingthe characteristic information of the received speech signal with thecharacteristic information stored as the registration information of theregistered user, execute the voice command and output a result ofexecuting the voice command, based on determining that the speaker whouttered the first text and the voice command corresponds to theregistered user, and ignore the voice command, based on determining thatthe speaker who uttered the first text and the voice command does notcorrespond to the registered user.
 5. The electronic device of claim 4,wherein the at least one voice command includes at least one of Internetshopping, financial transaction, call connection, message transmissionand alarm setting.
 6. The electronic device of claim 4, the at least oneprocessor is further configured, by executing the one or moreinstructions, to: output a synthetic speech signal generated byexecuting the voice command and display an image generated by executingthe voice command.